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Purpose: Tp interpret comprehension recalls in terms of textual- 
information avjd cognitive processes used. 1 * % - 

Date of Construction: 1981 j 

Physical Description: Whereas a recall is a "product" pf ' 
having read a text, it also results £rom' two sets of processes: 
receptive and productive. The instrument outlines 'four stages^ 
for. interpretive recalls. Stage 1 specifies which information^ 
is pertinent for analysis; stage 2 discusses the implications . 
of which linguistic unit to use in categorizing -the' data.;- The ' 
comprehension categories ai*e out,lineo\in stage 3; while 
stage 4 discusses the implications of Weighting of t;he. responses 
assigned to categories. ^ 

The procedures may be used with any recall.' The first category 
and the criteria for assigning responses to this category arg^ 
given below: . * y ' > A 



A. ' Text Exact . 

This category includes information from the text in its exact 
* form or with minimal variations. If is assumed that this in- 
formation was st'bred in rote fashion oi is automatically con- 
strained by other in-formation and is, "reproduced" in a 
similar state. x 

\ . - : • 

^Al. Verbatim Recall ... 

The ' information is a direct recall- of the lexical items of the 
t^xt. 



Text: The boys were late for school. 
Protocol: The bovs were late for school. , 

Substitution of a determiner, a verb fprm or a function word 
which does not change the meaning o.f the unit will aI*ro he 
placed in this categorv. ' u ' 

Text f He chased t^e animal. 
Protocdi: He chased an', ^n^mal. * 



Text: People were waiting at the door; 
Protocol: People were- waiting by the door. 

Text:: .The student'haQ been absent many times. . / * 
Protocol: The student .was absent many times. 

Partial Recall * . y. 
i -t . * * 

A significant concept (s) (noun, verb, attribute) is/are omi ttVd 

in the verbatim recall. 

Text^ After robbing the store^ the^ convicts 'raced for theirs car. 
Protocol: The* convicts raced for their car. 

1 / r | . -a 

Text: The children .had p^ver seen iuch a tiny colt. 
Protocol: The children had never seen such a colt. 

This category would also include, fragmen led units which ar^e not 
mazes and although not semantically complete dp indicate that, the 
reader has noted and attempted to retrieve concept s' which- cont inue 
the story line. r ^ 

"* . * ' 

Xext : The stranger told him to follow his advice and put his 

lines at- the spot indicated. 
Protocol:' The\ stranger told him... that he°would p ut . . . all : his • , 



lines . 



J 



Reliability and Normative Data: 

The comprehension categories have been b^sed on the construct of 
reading comprehension" (as measured' by a re^ll) as involving fhe 
'reception and production of information whicn is generated frrfm 
an interaction of the text data an$J the reader's prior knowledge. 
As indicated in the description of the... categories , certain 
assumptions about the underlying processes that, may contribute to 
> that, category are made based on the work of Kintsch and VanDijk 
(1978). - 

The ' categories may be sequenced in terms of the proportion of N text 
data' and prior knowledge that may have contributed to the recall. 
This sequence may be'* illustrated* by 'the following diagram with 
the amount of text decreasing from text exact to text experiential 




Text NjxMriential 
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Interrater reliability based on assigning -187 clause unitjerTo 
categories produced the following -percentages • of agreement . ' 



Category 



A 

B 

C 

D 

E 



98.5 
97-'* 1 
93 
96 

93' 
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^ hension and production'. Psychological Review . Vol/ 85,' 
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Comprehension Categories ' ' 
J . ' t| /- 5 . ■ * ' ^ . • . ^' ; 

The rollowing categories are based* largely on the work of Drum f and Lan taf f 
^(1977), earlier work by the author (Fagan, 1978, 1980) .and on two Research ' 
studies, ttiat tested the earlier category 'system (Brake, 1981; &eebe , Fagan and 
Maliclcy, 1981). Their purpose is to*"provide a structure~'to assess the degree 
of comprehension* as indicated by 'a recall protocol. This may be achieved ifn . 
four stages.. '■ ' - A ' . 1 

Stage 1: Eliminating Irrelevant Data \ ' 

.The first step is' to Isolate that information which will be analysed. In 
order to do this it is necessary fo eliminate two categories of data: mazes 
and recall conventions. • • '• , t 

X\ Mazes may be either of -four types. j 

Filled -Pause (Audible^'Noise) : This consis ts- of sounds which may be represented 
as 'ah, er, J um, efc. • 

Filled Pause (Interjection) This consists of words or phrases-which see^ to 
mark time for the speaker before Agoing" on to the next though^^Examples are 
well, I think, yes, let (me, see, wait a minute, etc. 1 
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Filled Paused (Repeat ) : This includes the ^repetition of words or parts of "words. 
.,He saw a .golf - a golf cart. • 
The little girl wa3 per - perturbed. 
The words or word parti spoken initially are classed as the Repetition since it 
is assumed that onqe the' speaker repeats, he end£ the pause s and continues to 
.complete the utterance . 

Correction /Edi t : This consists of a jumble of t words preceding ,a 'change in" . * 
direction of wfrat v the ^person was ^bout to say,* or t preceding a better choice 
of words . * - " 1 * ■ , " • ' 

He wanted to sell - to byy the golf bails. . 
" . The boy collected -golf ba^'ls in the - , around where - where he . r on the golf 
• course near where he, was lived. 5 ^ 

\ There will be only one instance of* a type >of hesitation pause/correction 

edit within a sequence. For example; if ,a word is repeated six times, it'is 
' one instance of a Filled" Pause '(Repeat) ; if several words are- used before the 
^child gets back in the Tight- track, - this is one instance of ai Correct ion /Edit . . 

1 - / \ 

j j j .ft v . 

< v . Recall conventions are concerned more with the narrating than with the 
" . actual content of, the text'. They may express a reader' 'limitations in not 1 
being able to Remember or may include vague generalizations which appear to be 
a cover-up for* lack of specific knowledge. .Following are examples of recall 
conventions. ' . . • 

Text-: ' (no specific referents) % ^ 

Protocol 11 We 11 it 1 * says that < 
"And in the second paragraph the story says...'" ^ * 
""That's all I dan. remember.. !t * r \ 

Protocol: "That was a good story"-' ■ I ' » ■ 

"I found it haW^to remember the part where, all the characters 

were introduced." u * ' 

* ■ * ." i • 

' ' T 

/ t * Also included are phrases 'used by the. reader to insert an. event 'in proper 

sequence d*ue to forgetting while recalling. j . . \ 

Protocol: Before that he set out his hookfe for fishing.- (The "befofe 

that" acts as an , addendum" t'o insert information in its proper 
sequence after subsequent information had beeri recalled. A 
synonymous statement to "before . that" would be "I forgot that 4 '. 

; . "• > . . ;> ..• • v . 

'Text: (detailing the advantages of heat) 1 , 

C ' • •■ . 

Heat helps lots. k 




scribing the manufacture of various 'items • of clothing) 
Protocol: t^iey make dresses / and stuff like that , 
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Stage- 2: Choosing' a Unit for Analysis - , • ■ : ' : 

Different units may be chosen for analysis proposition (Kintsch, 1974), 
syntactic proposition (Fagan, 1978) , "clause or t-ynit (Hunt, 1^65). An' 
assumption made when choosing a^unit is* that this represents a meanittgfujJ ■ • 
division of information and that the reader may perceive this.^unit when 
comprehending and/or ' recalling" information. Since it cannot^lie determined with, 
def initiveness which unit ^operates in 'this "manner (in fact: it to, likely that 
different units may be processed at different times) , the unit chosen for 
analysis will have different implications w^ien interpreting the results v - 

obtained,. For example, 1 ? if the smallest unit is chosen - the syntactrc^ro- 
position-.- then it is easier to determine if this "fit's into a category since 
^verbatimness , synonymy etc. is easier to analyse within this smaller' urtdt than 
within a larger unit such as the t— T^'f;.,. Consequently the interpretation of 
results would be weighted in t?er:r ' this category. If, on the other hand, . 
the t-unit is ^chosen then it is ier to judge if a summary has taken place^. 
since it is -.difficult to provide a s^snary of information within, the brevity 
of 'the syntactic proposition. Ine- clauiro. unit is intermediate in length b 
between the syntactic proposition and the t— unit/incomplete - t-unit , and ■ 
whereas it may ndt ha^e the full advantages of either of these, it also ; doej}J. 



not .have thei^r ^full' disadvantages. 

In orde,r to isolate clauses, it is suggested tha£ the protocol first be 
divided into t-unit s and ^incomplete t-units (Fagan, 1978) which are defined 
as follows: - ^ ' . 

T-unit : This is a single independent prediction (main clause) # together with 
an^ subordinate clauses that may be grammatically related to it. It may be a 
single or a- complex Sentence., but not a compound sentence. ■ 5 / 

In dividing a passage into t-units, one approach is to consider you are ' 
^editing the transcript andr'are directed.; to rewrite as sentences according to,, 
the - definition * v ab ove . ; Inhere there is a compound sentence^ divide before the 
connecting conjunction (and, but, etc.) and begin the next sentence with the 
conjunction. Do not change any words, but bracket these words whi"ch do not 
"fit into the regular flow of language tfrat make up the t-unitslie,, mazes). 

' - . * v 

Further guidelines for segmenting t-units are: 

tr 

1. When a quote consists of more than one principal clause, only the^ 
first one is included with the words that identify the speaker. 

e.g. /Christopher *said uncle when sha^l we get thj^re/it's 
such a long walk/ 

2. Having a t-unit within a t-unit is possible. 

e.g. /and he (/now he was scared/) told the captain . . . N 

3. vlhen the meaning of a passage indicates that a subordinate con? 1 

? junction has been omitted, the clause involved does ndt form a hew" 
t-unit. 

e.g. /he decided that he should go cause there was nobody around f 
and (cause) there was -stuff'. . .? 



/ 



4. M Yes tf fi§ included in the succeeding t-uni't if the: following state- 

• merit is an elaboration of the <an§wer ;--otherwise , it is considered 

to be an incomplete t-unit. ; 

{ ; / ; ' .'/ 

1 K , 5 e.g. /yes I guess yoti missed . . V/ ' 

/yes/ what do. you want it for/ yjr. . 

.5. Intonation may determine the location of the boundary when a phrase, 
•structurally , can be attached to either the preceding or subsequent 
t^unit . - 

e.g. ?i 4*thinlc ?f as in: , ^ 

* /he went I think/^B said he planned to any., way/ ^ ,V 

.6. * --Expressions like "I think, 11 !, I believe" are 'considered part of the - _ 
t7unit if they are integral to the statement as for example: 

J % \ ■ -r' ^) J 

/I think he went -saitf John/ 
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If the expression appeals, to be idi^osyntactic to the speaker, the 
' ./ ''words are considered a "folder 11 type maze and are not counted as 
part o£ the t-unit. * 

/■ v # e.g. ^Floods cause much damage/ (I think) ^ 

Incomplete t-unit : 'This consists of a gtoup of words which do nofc form a 
complete "independent clause but wTiich are 1 ne\*essary to the ongoing flow, 
langua'ge. Since*' it does not form a complete independent clause, it is^di f f erert t 
from a t-unit. It 'may. be lacking a subject, adverb, object, or complement or 
any combination, of these. * ' 

. - ■ ^ . r . 

' ".The incomplete 'appears to serve 1 either of four functions: specifying 
particular information; elaboration of an antecedent; making additional 
comments on a topic;. or establishing a referent"' for ■ an ambiguous iteflil^ 

He pushed one guy down in the water , J pushed him 'on 'the ground , started 
■"V * punching him . ^ v & ' 

So,her*-got *fed 'up with this kind of deal, everyone chasing him . — 

And so the man is looking", . couldn' t find him . ' ^ . 

He'd sell it to them, the 'balls , x , * 

. V ; . v 

An analyses of a transcription is given below. ' • 

He asks them for four golf bdllfr/'/pr he's gonna' put his boots into 
'the river/ /(and, and) (urn) (he gave* them) //the boy gave them four- 
(four) (fcm) golf balls//they. drop his boots anyway/because they are 
mean//he goes back looking for them/ /goes home {because after) (urn)// 
he had a dream// 7 . . • r 

T -'units // - 6 ' . • V 

Incomplete t-unit " 1 . * ■ ' * r 

'Filled -Pause (Audible noise) 3 <5 
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Filled Pause (Repeat) 
) 

Correction/Edit^ K _ 2' 

Clauses . ./ and ?. / / ' 1 

' ■ ^ S 
Sta_ge t 3: Comprehension. Categories 

A. Text Exact ' • ; / r „ 

v This category includes m information from the text in its exact, form or 
with -minimal variations. It is assumed .that this information' was 
stored in rote fashion ' or -is automatically constrainQ.d>b.y other, infor- 
O m v matior\ and- is reproduced in a similar state. / 

Al. Verbatim Recall \ - * 
• ./ * . f ' ■ . 

The information is a direct recall of the lexical items of the text. 



Text: - The bovs were late for school. 
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Erotocol: The boys were, late for school. '" ft % 

✓Substitution of a de terminer , a verb form or a function word which 
do£s±< not change the meaning of fhe unit will also hi placed in this 
^ category. * 

i Text:/"'He chased the animal? _ ' . * 

^ ?/ ! . ■ 

Protocol: <He chased an animal. 

^ - i ' 

Text: People were waiting -at the^door. 
* - Protocol:"'' People were waiting'by the t dodr. 

Text: The studeiy: had been absent many times, 
„ Protocol: The -student was absent many times. . v 

A2. Partial Recall >v 

= — \ ^ ^ «/ ( ^ 

*A significant concept(s) (noun, verb ,• attribute) is/are omitted ih the^ 
* ve-rbatim recall., " \ 

Text.: Aftjer fobbing the store, the convicts raced for their car* 
Protocc{l: The convicts* raced for their car. 

Text: The children had never seen such a tiny 'colt. # ' 
Protocol: The children h&d never seen such a celt. 

This category would also include f ragmeritad "units which are not mazes 
. and although not seiuant ically complete do indicate that 'the reader 
has noted and attempted to retrieve concepts which continue the^ 
story line . " - * 

y Text: The Jst ranger told him to follow his advics and put his lines 
at the spot indicated. ... ■ C \ 

Protocol:. The stranger told him. . . that he would put.. '.all his lines... 



J J. 
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••- Tp.xt Specific / 



, _ . r-. , , / 

In this category is placed information recalled that has specific 
preferences in the text. The ^reader may' have "transformed" some of this * 
t information- by reordering or substituting JLexical items. 

Bl.> Substitution of Prcnouns N ^ *, . // 

A. pronoun is used in place of a noun when the noun referent is present 

elsewhere within the.\text. Ml 'other items in the .Unit are'verbatim. 

* ■ A . / 
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Text: People were very* kin.^ to the stranger. % 
Protocol: They were very kind to the stranger- < m 

' Text: The truck went off the road about one half mile, from the \ 
feet t lament.. j 

Protocol: It went off , the *oa'd about one half mile from the settlement- 

, . - 7 \ . - ' < . 

B2 . Synonymy .of Elements * ^ 

..The operational definition of. synonymy is context dependent and may refer . 

, to -(a) substitution of one: word for* another so that semantic and gram- 
matical features are preserved , (b) the sequencing of lexical items from 
a unit such'as the preposing of prepositional phrases of substituting -an G 
active for a passive, and ( c) ' a paraphrase of the original unit which in < 
the"- subjective opinion of the scorer has the same conceptual . referents 
and has definite correlates' in the text unit. 

' \ ' - ' m% . ' /' ' , "~ ^ 

jText: fish m \ ■ . * 

'Protocol: salmon .~s * * „ . 

/ ' ' • > , • • ;'\ • • ' ; 

Text: The house was' on fire. - / x 

Protocol: The 'house was burning. • - ' * 

. ' " * ' ' / 

Text: In twos and very slowly the mourners walked in procession.; • 
t. * 

Protocolr The mourners walked' in procession very slowly and in twos. 

Text: He said good night and'went to' bed. . . t 

<* * * • 

Protocol: He decided to call it an evening and said good night. ^ 

C, Text Entailed 

The inf ormat ion-rr.etrieved is (a^ a- paraphrase of or synonymous with the. 
dnform^t^n input, but the-unit of recall^ncludes information from more 
than one unit of input, or (b) a superorditiate statement -subsuming infor- 
mation frgm Tnore than ( one text unit. It may be assumed; that at, the time 
. of ^comprehending the reader "constructed 11 information and may still . ' x -■"^ 
"transf ojrm 1 ' it at the point of recafl. 
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CI. Svivthesi's ' • * " 

■> ^ . - • " 

^ ? - A synthesis statement;; is (i) a compilation of at least two units of •• 

information. It may .not- contain either of the specific- units summarised 
v but may be 1 expresses in a hierarchical or superol*dinate category or by .a ' 
label generalizing 'the events summarized, such as a main idea, theme, or 
* ■ . moral. , ^ , 

, s ■ Text: *He quickly rac^d"" to the landing, stripped off his clothes and 
jumped Into the icy water to rescue the frightened little, boy. 

Protocol: He did a very brave deed. . s 

" i * 

Text: While ' visiting her Aunt Lizzie at the 'farm' last weekend; Teri ' 
t helped harvest some carrots, peas, zucchini and tomat6es v * , 

Protocol: ' Last weekend, .Teri helped her- Aunt harvest some vegetables. 

\ . » & . '' t / 

C2. Summary '' . <• / f 

• ■ v V : 

A statement is a summary if it relates information from at le"ast two 
units <in the text in an embedded form, t-hat is some of the lexical 
items or units of information are deleted during this process. In 

summarizing the exact vjprds ■ or their synonyms may be used. . ; 

' ■ ' t " 

Text: She jumped, into the icy water. She was trying to §ave the 
swimmer who was in t rou kle. 

Protocol:- She jiumped into the icy water to save swimmer in trouble. 

Text: The stranger pitied the man. life had tried to help but had not 
been very successful. The stranger felt deep remorse but knew 
that the man wo$ld have to settle his own problems without out- 
side interference.- The stranger stared quietly as v the man < 
walked slowly away. \ ( * r ~^- 

Protocol: . The stranger pitied the man who walked slowly away. 

D. » Text Experiential ' / * 

1 ' .<'.'*'• 

This information is added by the reader *to fill in 'gaps in the text 

data. The" reader is, 11 reconstructing" ' information based on prior know- 
ledge, which may' be of world events such as rodeo, or from having read 
or listened td other texts. ' 

Dl. Inference > . • .. 

An inference may include ,either a logical* reasoning or an instantiation, 
-tjiat is, the filling in of information suggested' by the text ' inf ormat i on 
* but not specified. The latter is often referred tb~as a pragmatic 
inference and may be .stated in a contradictory form and still make a 
plausible statement. v , 

Text: John and Bill left for schobl at the same time ana. va Iked at the 
same rate,. But Bill lived several blocks - farther away from the 
school than John. * John just reached- the. school on time. He \ 
h.oped that Bill .would still be .able to play ball that evening. 

Protocol: (Logical)*: % Bill' was .late for school. ^ ' .„ 



Text: The motrher -bundled the chil'd'ren in their parkas, scarves ancl 
' . <• mittens. She was sure they all .'had a hot lunch as they left « 

for school. - f ' \ " 

» 

Protocol: (Pragmatic): It was .a cold day. , 

(Contradiction: * It was not a cold day. Perhaps the mother 
was mentally deranged) . 

D2. 'Case Related^ Information 

This includes the' expansion of permissible sequences that are assumed 
! extensions of a unit of information in the text. This subcategory 

describes appropriate prior knowledge of similar content. 

/ 

Text: Ground corn 

Protocol: Ground corn with a rock. . 

Text': The captain climbed the mast of the distressed ship and signalled , , 
■ . . f or 'help . 

i 

Protocol: The captain climbed the mast of the distressed ship and) 
7 signalled for help with his flag. 

-***V . - 

Text: / Used 'for etching. 

"> » 
Protocol: Used for t^he etching of drawings. 

D3.. Experiential Intrusions 

This information is related to - 4 the theme of the text passage but is not 
specifically suggested by a particular unit in t^e text. It does not 
0 convey the text information but is an addition of information from the 
reader's background. 

Text: The little boy had disobeyed his mother. She had told him to 
wait by the car Xsfh'ile she went back to the store for the other 
bag of groceries. Now she could not find him anywhere. 

Protocol:. One t^ime I saw this woman looking everywhere for her little 
bt>y. He vent up the escalator wher\, she wasn't looking. 

D4<. Storyline additions ^ } 

These, units include additions to the information within the storyline. 
The origin of these additions appears to be based cn the reader's 
experience with stories and the x kinds of goals or actions ■ which are 
appropriate in a particular context and thus are predictable from the < 
story inclination. Also included are expressions that indicate saying, 
thinking, etc., whfch are not. specifically stated in the text. These 
are not inf erence*s v **since they are not immediately constrained by a 
specific *, part of the text. " : v , 

. s Text: (describing a characters actions that 'let up to making a decision). 

Protocol: He thought he ^woujLd catch the nf;xt train and finally settle 
the matter complepely. 
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Text: The stranger saw that the man was weak and finally dug, a hole., 
through the ice for him. 

Protdcol: The man said tf I am not able to dig the hole. 11 But the 

stranger said M You g'ot to keep trying and trying." The 

man said L, I just can't do' it. 11 \ \ 

\ * •* ; 

Text Erroneous . ' 
" . & 

The protocol units' involve the use of text information which the reader 
has processed incorrectly either at the time of comprehending or at 
production /of the recall. ' '* \ 

Er rors in dates and proper names i / 

These errors constitute memory errors or are due to 'lack of Attention 
to the text. The appropriate slot is there but is inaccurately filled. 

'Text: Sir Wilfred Laurier 

Protocol: Sir Wilfred Bennett ' , 

Text: 186 A = 
Protocol: 1872 ' 

Erroneous expansions /additions \ 



These units (i) separate attribute/argument phrases into units* that are 
conceptually wrong, (ii) expand a. unit of information in an erroneous 
way (D2) , or add information that is incorrect in terms of world know-* 
ledge of the events mentioned, or is contradictory • with information in 
the text. These may be due to lack of experience with the content and/or 
the ambiguity of the text. ' 

,Text: They ground corn. 
Protocol: They ground- corn by heating it. 

..J 

Text: The lobster's claws.' 
Protocol: The lobster claws. 

Inaccurate /incorrect synthesis 

Information from different units of the text- is (i) designated by ajn 
inaccurate superordinate referent, ot (ii) is generalized in a way which 
does not convey the gist of the passage. 

/ 

Text: We shouldn^ t always knock computers when they seem to make an 

error on pur accounts. Granted we might be upset when our balance 
is nil arid the computer still insists that we send a check for 
$40.00. However, if computers were assigned to dcx the many menial ■ 
tasks of administrative affairs and leave more time^ f or humans to 
use tbrelr intelligence to solve the more significant problems, then 
computers and humans would"\be compatible and would coexist in harmon) 

Protocol: Computers are frustrating. 

% • 



r 



/ 

1.4 , 



11 - 



Text: While visiting her Aunt Lizzie atj • the farm last weekend, Teri ^ 
helped harvest some carrots, peas, zucchini j vand tomatoes. 

.Protocol: Las*, weekend Teri helped her Aunt harvest some fruit. 

E4. -Inaccurate/incorrect summary 

In combining information the reader confuses information about a par- 
ticular referent. v 

TeA: As" the man was scraping snow off the ice he saw someone standing 
beside him. The man said to the stranger "I don't think I can 
finish visiting my lines because I am so cold and hungry." The 
stranger said he would help. He dug new holes for the man and 
also showed him where to get caribou. 

Protocol: A ^stranger came a'long. He helped the man dig holes through 
the ice- and then they saw a caribou herd go by. 

' Text: Mrs. Gray sat down > to watch the TV announcer on her weekly show 
about gardening. J 

. Protocol: Mrs. Gray sat^down to "watch the TV announcer on his weekly 
-y show about gardening-. ^ 

Text: r The dogs lay down and refused to move. The man dragged the sled 
all the way to the cabin. 

Protocol: The, dogs dragged the sled to the cabin. 

E5 . / Faulty Inference : 

The reader draws an incorrect inference from the information given in 

the text. ' , m 

/ 

Text: Mrs. Gray knew it was two o'clock because she could hear Henry, 

her parrot squawking. He wanted to watch his favorite TV program. 
But Mrs. Gray thought that too much TV was bad for Henry's eyes so 
she told him to rest insteadT He squawked even louder so she 
finally turned on the TV set. After Henry's show was over, she 
stayed to watch a show on cooking. * N ' 

Protocol: Mrs.. Gray came in from the garden to watch her TV show. 

Stage 4: Weighting 

The matter of assigning *a weight in points to the unit chosen ±s an 
arbitrary decision and should be determined by the purpose of the analysis. 

It is suggested that the weighting be assigned on the basis of the number 
of categories -evident in the reader's recall as opposed to the numbej of' units 
recalled. That is, one unit may be assigned to two categories. The rational- 
ization for this is that the analysis is hopefully indicative of what' the 
reader was doing when reading. Consequently if one unit (clause) ^ind^Tcates 
that the reader used a Pronoun Substitution (Bl) and Case related Information 
(D2) , then this should be so noted. 
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fu\ incomplete t-unit is sometimes not a clause (lacks a finite' subject 



or 



verb) but is considered equivalent to a clause unit for scoring purposes. 



Validity and Reliability 



The comprehension categories have been based on the construct of reading 
comprehension (as measured by a recall) as involving the reception and gro^ * 
duction of .information which is generated from an interaction <of the text data 
and the reader's prior knowledge. As indicated in the description' of the 
categories, certair assumptions about the underlying processes that may con- 
tribute to N that . categorv are made based on the work of Kintsch and.VanDijk 
(1978). N ' * ' 

The categories may \>e sequenced in terms " r of the ' proportion of te^t data 

and prior knowledge that m&y have contributed to the recall. This\ sequence 

may be illustrated by the following diagram with the amount of text decreasing 
from text exact to text experiential. 




Text Njxoerient ial 



In order to* obtain adequate reliability , the following guidelines are 
suggested; v -s. * 

1. Each scores be thoroughly familiar with the categories,, their, 
definitions, and examples^ 

2. A number (of protocols be scored as points for discussion before 'the 
independent scoring is done. At this point, definitions, or 
examples may need to be further clarified. , 



3. If »a unit\ is not readily assigned to a category, then the scorer 
should engage in the process of category elimination. 

•* t • 

4. Posing a question on the unit being analysed may help clarify the 
category "which represents what the reader(was doing. For example, A 
if the unit, supposedly indicates Text Verbatim, then an appropriate j 
question for the reader is "Tell me if this was present in what you } 
just read. 11 If the unit is suspected as/ being Case Related Infor- 

' mation, then a question might be "Is this true about grinding corn? i 
-Do they pound it with a stone? 11 For an. inference the question pose^f 
might be "What information in the story suggests this statement (the 
inference)/?" For a synthesis, the question "Can you elaborate on 
this?" might be considered. The scorer will have to judge whether br 



.not the reader could respond to such' questions . If so, then it is 
plausible that this unit belpjvgs" within the category indicated. * 

Using the above guidelines, five recatl^pjxftocols comprising 187 clauses 
were .assigned to categories. The interrater reliability 'in terms of per- 
centage agreement were: * . 



Category A 


98.5 


• B 


' . 97 


C 


" 9 3 




•96 


E 


95 



Concluding Statement " *f 

) 

-In order to assess 'comprehension , one must consider both the process 
(reception and production) and the product. It is difficult to get at the 
former which must be inferred. Th$ comprehension categories provided in 
this article hopefully will allow both factors to be taken into account. 



References ,■ v 

Brake, M.D.. Oral and silent reading of low and high achievers. Unpublished 
Masters Thesis, The University of Alberta, Edmonton, 1981.- 

Beebe, Mona J. , Fagan , William T., and Malicky, Grace V. Discourse processing 
operations' and reading. Unpublished paper. The University of 
j Alberta, 1981. * . M 

Drum, P. A. and Lantaff , R.E.- Scordng categories for protocols. Paper pre- 
sented .at the Second Annual Lan^u^ge Conference , Boston ,. 19 77 . 

Fagan, William T. A comparison t or the oral language of children aged nine , 
ten and eleven , A research report. Canada Council grant 
S76-0563. Department of Elementary Education, The University 
of Alberta, Edmonton, 1978. ' ■ 

Fagan, William T. Comprehension categories for protocol analysis. Unpub- 
lished paper. The University of Alberta, 1980. 

Fredericksen, Carli Representing logical and semantic structure of knowledge 
acquired from discourse. Cognitive*- Psychology . Vol; 7, 1975a, 
317-458. ; v. ' ' v - 

Fredericksen, Carl. Understanding discourse.: what knowledge is acquired and 
how. Paper presented at Bell Telephone Laboratories, 1975b., . • 

Guzak, Frank. Teachers 1 questions and levels of reading comprehension. 

Perspectives in Reatiingf The Evaluation of : Children 1 s- Reading 
Achievement . Newark, Delaware: International Reading 
Association* 1967. r 

Hunt, Kellogg W. Grammatical structures written at three grade levels .^ 
Urbana, Illinois: National Council of Teachers of English, 
1965. " . - 

Kintsch, W. The representation of meaning in memory . New Jersey: Lawrence 
• Erlbaum Associates, 1974. 

Kintsch, W. and Keenan, J. Reading rate and retention ^as a function of the ^ 
number of propositions in the base structure of the' sentences. 
. Cognitive Psychology . Vol. 5, 1973, 257-274. ' 

Kintsch, Walter and -VanDijk, Teun A, Toward a model^of text comprehension 

and production. Psychological Review . '^Volt 85, No. 5, 363-394, 
. 19 78. J 

Mandler, J.M. , and Johnson, U.S. Remembrance of things parsed: . story .. 

structure ' and recall. Cognitive Psychology . Vo. 9, 111-151, 
,1977. • 

•* . * 

Stein, N.L. How children understand stories: a developmental analysis. 

Center for the Study of Reading, University of Illinois, Urbana, 
1978. * f 



ERIC 



